Using blind analysis for software engineering experiments

Context: In recent years there has been growing concern about conflicting experimental results in empirical software engineering. This has been paralleled by awareness of how bias can impact research results. Objective: To explore the practicalities of blind analysis of experimental results to reduce bias. Method : We apply blind analysis to a real software engineering experiment that compares three feature weighting approaches with a na ̈ıve benchmark (sample mean) to the Finnish software effort data set. We use this experiment as an example to explore blind analysis as a method to reduce researcher bias. Results: Our experience shows that blinding can be a relatively straightforward procedure. We also highlight various statistical analysis decisions which ought not be guided by the hunt for statistical significance and show that results can be inverted merely through a seemingly inconsequential statistical nicety (i.e., the degree of trimming). Conclusion: Whilst there are minor challenges and some limits to the degree of blinding possible, blind analysis is a very practical and easy to implement method that supports more objective analysis of experimental results. Therefore we argue that blind analysis should be the norm for analysing software engineering experiments

ICE Second Halley radial: TDA mission support and DSN operations

The article documents the operations encompassing the International Cometary Explorer (ICE) second Halley radial experiment centered around March 28, 1986. The support was provided by the Deep Space Network (DSN) 64-meter subnetwork. Near continuous support was provided the last two weeks of March and the first two weeks of April to insure the collection of adequate background data for the Halley radial experiment. During the last week of March, plasma wave measurements indicate that ICE was within the Halley heavy ion pick-up region

Frequentist and Bayesian measures of confidence via multiscale bootstrap for testing three regions

A new computation method of frequentist $p$-values and Bayesian posterior probabilities based on the bootstrap probability is discussed for the multivariate normal model with unknown expectation parameter vector. The null hypothesis is represented as an arbitrary-shaped region. We introduce new parametric models for the scaling-law of bootstrap probability so that the multiscale bootstrap method, which was designed for one-sided test, can also computes confidence measures of two-sided test, extending applicability to a wider class of hypotheses. Parameter estimation is improved by the two-step multiscale bootstrap and also by including higher-order terms. Model selection is important not only as a motivating application of our method, but also as an essential ingredient in the method. A compromise between frequentist and Bayesian is attempted by showing that the Bayesian posterior probability with an noninformative prior is interpreted as a frequentist $p$-value of ``zero-sided'' test

Virial Masses of Black Holes from Single Epoch Spectra of AGN

We describe the general problem of estimating black hole masses of AGN by calculating the conditional probability distribution of M_BH given some set of observables. Special attention is given to the case where one uses the AGN continuum luminosity and emission line widths to estimate M_BH, and we outline how to set up the conditional probability distribution of M_BH given the observed luminosity, line width, and redshift. We show how to combine the broad line estimates of M_BH with information from an intrinsic correlation between M_BH and L, and from the intrinsic distribution of M_BH, in a manner that improves the estimates of M_BH. Simulation was used to assess how the distribution of M_BH inferred from the broad line mass estimates differs from the intrinsic distribution, and we find that this can lead to an inferred distribution that is too broad. We use these results and a sample of 25 sources that have recent reverberation mapping estimates of AGN black hole masses to investigate the effectiveness of using the C IV emission line to estimate M_BH and to indirectly probe the C IV region size--luminosity (R--L) relationship. We estimated M_BH from both C IV and H-Beta for a sample of 100 sources, including new spectra of 29 quasars. We find that the two emission lines give consistent estimates if one assumes R \propto L^{1/2}_{UV} for both lines.Comment: 38 pages, 6 figures, accepted by Ap

Why Ni3_3Al is an itinerant ferromagnet but Ni3_3Ga is not

Ni$_3$Al and Ni$_3$Ga are closely related materials on opposite sides of a ferromagnetic quantum critical point. The Stoner factor of Ni is virtually the same in both compounds and the density of states is larger in Ni$_3$Ga. So, according to the Stoner theory, it should be more magnetic, and, in LDA calculations, it is. However, experimentally, it is a paramagnet, while Ni$_3$Al is an itinerant ferromagnet. We show that the critical spin fluctuations are stronger than in Ni$_3$Ga, due to a weaker q-dependence of the susceptibility, and this effect is strong enough to reverse the trend. The approach combines LDA calculations with the Landau theory and the fluctuation-dissipation theorem using the same momentum cut-off for both materials. The calculations provide evidence for strong, beyond LDA, spin fluctuations associated with the critical point in both materials, but stronger in Ni$_3$Ga than in Ni$_3$Al.Comment: replaced (incorrect version submitted

A Nonparametric Method for the Derivation of α/β Ratios from the Effect of Fractionated Irradiations

Multifractionation isoeffect data are commonly analysed under the assumption that cell survival determines the observed tissue or tumour response, and that it follows a linear-quadratic dose dependence. The analysis is employed to derive the α/β ratios of the linear-quadratic dose dependence, and different methods have been developed for this purpose. A common method uses the so-called Fe plot. A more complex but also more rigorous method has been introduced by Lam et al. (1979). Their method, which is based on numerical optimization procedures, is generalized and somewhat simplified in the present study. Tumour-regrowth data are used to explain the nonparametric procedure which provides α/β ratios without the need to postulate analytical expressions for the relationship between cell survival and regrowth delay

Gross Job Flows in Ukraine: Size, Ownership and Trade Effects

This paper documents and analyses gross job flows and their determinants in Ukraine using a unique data set of more than 2200 Ukrainian firms operating in both the manufacturing and the non-manufacturing sector for the years 1998-2000. There are several important findings in the paper. Job destruction is dominating job creation in both 1999 and 2000. In connection with other evidence we infer from this that Ukraine is only at the beginning of the restructuring process. The most clear-cut result of our analysis is the strong positive effect of new private firms on net employment growth, a finding established for other transition economies as well. At the same time, we do not find differences in the employment growth of state-owned and privatised firms. Apart from ownership effects we also find, at the firm level, an inverse correlation of size and net employment growth and of size and job reallocation. Finally, we establish that strong foreign trade links force firms to shed labour more aggressively and to engage in more restructuring when trade is directed to and originating from Western economies. This disciplining function is absent when the trade flows are confined to CIS countries.http://deepblue.lib.umich.edu/bitstream/2027.42/39906/3/wp521.pd

Critical behavior of the Random-Field Ising Magnet with long range correlated disorder

We study the correlated-disorder driven zero-temperature phase transition of the Random-Field Ising Magnet using exact numerical ground-state calculations for cubic lattices. We consider correlations of the quenched disorder decaying proportional to r^a, where r is the distance between two lattice sites and a<0. To obtain exact ground states, we use a well established mapping to the graph-theoretical maximum-flow problem, which allows us to study large system sizes of more than two million spins. We use finite-size scaling analyses for values a={-1,-2,-3,-7} to calculate the critical point and the critical exponents characterizing the behavior of the specific heat, magnetization, susceptibility and of the correlation length close to the critical point. We find basically the same critical behavior as for the RFIM with delta-correlated disorder, except for the finite-size exponent of the susceptibility and for the case a=-1, where the results are also compatible with a phase transition at infinitesimal disorder strength. A summary of this work can be found at the papercore database at www.papercore.org.Comment: 9 pages, 13 figure

sscMap: An extensible Java application for connecting small-molecule drugs using gene-expression signatures

Background: Connectivity mapping is a process to recognize novel pharmacological and toxicological properties in small molecules by comparing their gene expression signatures with others in a database. A simple and robust method for connectivity mapping with increased specificity and sensitivity was recently developed, and its utility demonstrated using experimentally derived gene signatures. Results: This paper introduces sscMap (statistically significant connections' map), a Java application designed to undertake connectivity mapping tasks using the recently published method. The software is bundled with a default collection of reference gene-expression profiles based on the publicly available dataset from the Broad Institute Connectivity Map 02, which includes data from over 7000 Affymetrix microarrays, for over 1000 small-molecule compounds, and 6100 treatment instances in 5 human cell lines. In addition, the application allows users to add their custom collections of reference profiles and is applicable to a wide range of other 'omics technologies. Conclusions: The utility of sscMap is two fold. First, it serves to make statistically significant connections between a user-supplied gene signature and the 6100 core reference profiles based on the Broad Institute expanded dataset. Second, it allows users to apply the same improved method to custom-built reference profiles which can be added to the database for future referencing. The software can be freely downloaded from http://purl.oclc.org/NET/sscMapComment: 3 pages, 1 table, 1 eps figur